DOCUMENT RESUME 



ED 427 675 



IR 019 194 



TITLE 

SPONS AGENCY 
PUB DATE 
NOTE 



CONTRACT 
AVAILABLE FROM 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



A Smart Curriculum for Middle-School Science Instruction: A 
Web-Based Curriculum Integrating Assessment and Instruction. 
National Science Foundation, Arlington, VA. 

1996-10-00 

9p . ; In: WebNet 96 Conference Proceedings (San Francisco, 

CA, October 15-19, 1996); see IR 019 168. Figures may not 

reproduce clearly. 

MDR- 9252908 
Web site: 

http : //aace .Virginia . edu/aace/conf /webnet/html/305 . htm 
Reports - Descriptive (141) -- Speeches/Meeting Papers (150) 

MF01/PC01 Plus Postage. 

♦Computer Assisted Instruction; * Computer Managed 
Instruction; Computer Mediated Communication; Courseware; 
Educational Technology; Feedback; Intermediate Grades; 

Junior High Schools; ^Middle Schools; Models; *Multimedia 
Instruction; *Multimedia Materials; Science Curriculum; 
♦Science Instruction; World Wide Web 

♦Alternative Assessment; Virtual Classrooms; Web Sites 



ABSTRACT 



This paper discusses a model of integrated instruction and 
assessment called SMART (Special Multimedia Arenas for Refining Thinking) . 
SMART involves interactive use of the Internet and multimedia software. The 
Internet serves three important functions: it acts as a formative assessment 
tool by providing individualized feedback to students, creates a learning 
community by displaying data submitted by participating classrooms, and 
promotes discussion and reflection on important concepts. This paper 
discusses design features of the SMART model, including the Internet tools, 
and describes an example from middle-school science. Two figures present an 
excerpt from the SMART WWWeb macroinvertebrate catalog and an excerpt of 
student feedback from SMART WWWeb. (Author/DLS) 
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Abstract: This paper discusses a model of integrated instruction and assessment called SMART 
(Special Multimedia Arenas for Refining Thinking). SMART involves interactive use of the 
internet and multimedia software. The internet serves 3 important functions: It acts as a formative 
assessment tool by providing individualized feedback to students, creates a learning community 
by displaying data submitted by participating classrooms; and promotes discussion and reflection 
on important concepts. We discuss design features of the SMART model, including the internet 
tools, and describe an example from middle-school science. 



A primary focus of current research on alternative assessment involves the investigation of performance 
assessments for summative purposes, in particular for school- and system-level accountability [Baker & 
O'Neill 1994]. In this context, researchers have largely been concerned with the psychometric properties 
of performance assessments, notably, with issues that have arisen with respect to scoring, reliability, and 
validity. 

While these issues are important, this paper concerns a different, often overlooked, function of 
assessment— to inform ongoing instruction and learning. There are several reasons for our focus on 
formative assessment 1], One derives from our goal to transform classrooms into exciting learning 
communities that encourage students to achieve high levels of learning. We assume that teachers will 
make better instructional decisions by engaging in formative assessment, leading to improved student 
learning. We also assume that students will learn more and learn more deeply if they routinely engage in 
reflection and revision [Brown 1987]. 

In addition, formative assessment is fundamental to the new standards for instruction recommended by 
groups such as the [National Research Council 1996] and the [National Council of Teachers of 
Mathematics 1989]. These standards, based on social constructivist principles of learning, emphasize the 
importance of teaching in ways that promote deep understanding by students. Learning is no longer 
viewed as the accretion of new information. Instead, it is viewed as transformational process wherein 
concepts slowly evolve. In this milieu, teachers are being asked to adopt a more "cognitive" stance to 
teaching, for example, to be aware of the preconceptions that their students bring to new learning 
situations, to teach in ways that make students' thinking "visible" to other students and to help students 
reflect on and reconcile their conceptions with those of others. Formative assessment is an important part 
of this new repetoire of teaching behaviors. 

In this paper we discuss a model of integrated instruction and assessment that we call SMART (Special 
Multimedia Arenas for Refining Thinking). SMART involves interactive use of the internet and 
multimedia software for assessment and instruction. In SMART the internet functions as both a teaching 
tool and an assessment tool. We first describe design features of the SMART model, and then discuss an 
example from the area of science. In this context we elaborate on details of the tools we developed for 
the internet. 
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[1] We use the term "formative assessment" to refer to reflective practices by both teachers and students. 
Formative assessment by students is synonymous with self-assessment activities where students reflect 
on their conceptions. Formative assessment by teachers involves ongoing monitoring of students' 
knowledge and skills for purposes of instructional decision-making. 

EVOLUTION OF THE SMART MODEL 



The current SMART model derives from close to 1 0 years of research with students and teachers on 
ways to motivate and assess exceptional learning [Barron, Vye, Zech, Schwartz, Bransford, Goldman, 
Pellegrino, Morris, Garrison, & Kantor 1995] [Cognition and Technology Group at Vanderbilt 1994]. 
Our initial work focused on mathematics, and was concerned with an approach to instruction that we call 
"anchored instruction." In anchored instruction, teaching and learning are focused around complex 
problems or "anchors." The anchors are stories on videodisc (or CD-ROM) that each end with a 
challenge to solve. All of the data needed to solve the challenges are contained in the stories. The 
problems (a) are complex and require extended effort to solve (at a minimum, in the range of 3-5 hours 
for most middle school students); (b) are relatively ill-defined and require significant formulation prior 
to solving; and (c) have multiple viable solutions. The anchors are designed to engage students in 
authentic problem solving activities that highlight the relevance of mathematics to the world outside the 
classroom. 



Our initial research on anchored instruction was conducted using "The Adventures of Jasper 
Woodbury", a series of video anchors developed by members of our Cognition and Technology Group at 
Vanderbilt. Findings indicated that working on multiple Jasper anchors over the course of a school year 
resulted in significant improvements in fifth and sixth grade students' problem formuation and problem 
solving skills. In addition, students showed positive changes in their attitudes towards mathematics 
[Pellegrino, Hickey, Heath, Rewey, Vye, & Cognition and Technology Group at Vanderbilt 1991]. 
Nonetheless, reports from teachers and students were unanimous in their strong dislike for our 
assessments (these assessments were conducted as part of our research and consisted of traditional paper 
and pencil mathematics story problems). 



In thinking about how to re-design our assessments, we focused on assessment as it occurs outside of 
school settings. This was a valuable thought experiment in that it pointed to some important differences 
between assessment in and outside of school. First, in contrast to assessment in schools, assessment in 
professional contexts is usually external, and the products that are assessed are designed to contribute in 
some way to the profession. For example, when we write a paper or prepare a proposal for a 
presentation, our work is examined by expert individuals who are external to our department. Further, 
evaluation is not the sole purpose for generating products. Hopefully, papers and presentations 
contribute to knowledge, research and development in the field. In designing SMART we have tried to 
emmulate these features. Students' learning is directed toward culminating challenges that are evaluated 
by experts and have tangible consequences. For example, in some of our early work, students from 
different classes participated in live satellite programs in which they responded in real time to challenges 
related to Jasper [Kantor, Moore, Bransford, & Cognition and Technology Group at Vanderbilt 1992]. In 
recent work, the culminating challenges relate to project activities that follow Jasper. In one of the Jasper 
anchors students learn to design blueprints and a scale model of a playground and playground 
equipment, and in the project that follows, students are challenged to design blueprints and a scale model 
of a playhouse for kindergarten-aged children. Students present their designs to expert builders, and 
designs that meet prespecified evaluation criteria are entered into a random drawing. Designs selected 
during the drawing are built and donated to local kindergarten classes. [2] 



Another way in which assessment in school and professional settings differs relates to opportunities that 
are available for improving one's work. In professional settings, there is a comittment to creating the 
very best product that is possible. We solicit input from people both internal and external to our 
organization, and we pay careful attention to performance standards set by experts in the field. We rely 
on this information as we draft and re-draft our work; reflection is a critical part of the process. 
Regrettably, in most classrooms, opportunities for feedback, reflection, and revision are almost 
non-existent. When students do receive feedback, it is usually in the form of a grade— rather than 
something that could help them enhance their understanding— and opportunities to improve their work 
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are rare. 

To promote self-assessment and reflection in SMART classrooms, instruction is explicitly organized 
around cycles of work and revision, and we have designed technology-based tools to that provide 
feedback to students and help them improve their work. Our research indicates that students who use 
these tools learn significantly more than students who go through the same instructional sequence for the 
same 

amount of time, but who do not use the tools [Barron et al. 1995]. Initially, our tools consisted of 
videodisc programs and stand-alone computer applications. More recently, we have used the internet. In 
the sections below, we describe a just-completed experiment using our SMART WWWeb. 



[2] Space constraints preclude a comprehensive review of our research on SMART. We refer interested 
readers to [Barron, et al. 1995] and [Cognition and Technology Group at Vanderbilt 1994] for more 
information. 

THE WEB AS A TOOL FOR TEACHING AND ASSESSMENT 

As mentioned, in SMART students iterate through cycles of problem solving and revision. Students 
access our internet site, SMART WWWeb, during the revision phase. Essentially, SMART WWWeb 
serves 3 functions: First, it provides individualized feedback to students. In this way, the Web serves as a 
formative evaluation tool. The feedback suggests aspects of students' work that are in need of revision, 
and classroom resources that students can use to help them revise. The feedback does not tell students 
the 'right answer.' Instead, it sets a course for independent inquiry by the student. The Web feedback is 
generated from data that individual students enter. Essentially, data that is submitted by students in the 
brower is collected in a database on our server. Responses in the database are subsequently tagged with 
feedback that is sent back to the browser for students to print out. 

The second function of SMART WWWeb is to collect, organize and display the data collected from the 
distributed classrooms. Data displays are automatically up-dated as new data are submitted to the 
database by students. We call this section, SMART Lab. The data in SMART Lab consist of students' 
answers to problems and explanations for their answers. Each class' data are displayed separately from 
the distributed classroom's data. This feature enables the teacher and her/his class to discuss different 
solution strategies, and in the process, address important concepts and misconceptions. These 
discussions provide a rich source of information for the teacher on how her/his students are thinking 
about a problem, and are designed to stimulate student reflection as well. 

The third section of SMART WWWeb is Kids Online. Kids Online consists of explanations by 
student-actors. The explanations are text-based with audio narration. Still pictures of the presenters are 
also available. The explanations are errorful by design. Students are asked to critically evaluate the 
explanations, 

and provide feedback to the student-actor. By including errors we are able to seed thinking and 
discussion on concepts that are frequently misconceived by students. At the same time, students learn 
important critical evaluation skills. 

AN EXAMPLE FROM SCIENCE 

Our current work on SMART is focused around a video anchor on CD-ROM entitled, 'Stones River 
Mystery' (hereafter SRM). SRM is an episode in the series "Scientists in Action" developed by Bob 
Sherwood and his colleagues at Vanderbilt [Sherwood, Petrosino, Lin, Lamon, & Cognition and 
Technology Group at Vanderbilt 1995]. SRM tells the story of a group of high school students who, in 
collaboration with a biologist and hydrologist, are monitoring the water in Stones River. The video 
shows the team visiting the river and conducting various water quality tests. (In our work we focus on 
benthic macroinvertebrate sampling and dissolved oxygen testing.) Students in the classroom are asked 
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to assess the water quality at a second site on the river. They are challenged to select tools that they can 
use to sample macroinvertebrates and test dissolved oxygen, to conduct these tests, and to interpret the 
data relative to previous data from the same site. 

When students begin working on macroinvertebrates, they are given a catalog of sampling 
tools/instruments. Many of these are "bogus" and collect the wrong kind of sample; others are 
'legitimate' and will gather a representative sample of macroinvertebrates. The catalog items are 
specially designed to include contasting cases that help students discover the need to know certain kinds 
of information. For example, to use macroinvertebrates as indicators of water quality, one needs to 
collect a sample that represents the river's biodiversity, and as such, all types present need to be sampled. 
The following is the actual description of one of the items in the catalog, the TetraBen Laser Counter: 

"Knowing the number of macroinvertebrates in your water is an important way to determine the health 
of your river. Collecting and counting these organisms can be a slow, tedious process. Modem science 
has revolutionized this process. The TetraBen Laser Counter lets you count macroinvertebrates without 
getting your hands wet! Simply scan the laser beam slowly over the water. The laser beam automatically 
counts the macroinvertebrates, and shows the total number on a built-in screen. The laser is completely 
waterproof and won't harm anything, living or non-living (and that includes macroinvertebrates and 
humans!) Simple, safe, and completely accurate!" 



This is an example of an item that would collect the wrong kind of information; it counts the 
macroinvertebrates and ignores information about the types of macroinvertebrates in the sample. 

Students are asked to choose and justify their choice of tool. To help them make their choices, they are 
provided with resources, some of which are on-line, that they can use to find out about river ecosystems, 
macroinvertebrates, and water quality monitoring. Once students have made an initial set of choices, 
they use SMART WWWeb. They enter their catalog choices (yes or no), and select justifications for 
their choices (why or why not choose that catalog tool). Figure 1. shows a portion of the internet "order 
form." 
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Figure 1: Excerpt from SMART WWWeb macroinvertebrate catalog 



An important feature of the catalog and internet order form is that they are designed to reveal common 
misconceptions about ecosystems and pollution. For example, our pilot research on science showed that 
many students think that bacteria is harmful and pollutes rivers. We have tried to include catalog items 
and foils that expose particular misconceptions. In our macroinvertebrate catalog, we have a 'Super 
Collector Cone' that promises to collect macroinvertebrates and bacteria, and if students decide to order 
the Cone, one of the justifications that they can use is that 'bacteria pollute the water so it is important to 
catch them.' 



Once students have submitted their catalog order on-line, SMART WWWeb sends them individualized 
feedback. Figure 2. shows a segment of SMART WWWeb feedback. 
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Figure 2: Excerpt of student feedback from SMART WWWeb 

The feedback that students receive from SMART WWWeb highlights why the selected tool is 
problematic, and suggests helpful resources (sections of on-line and off-line resources, hands-on 
experiments, and peers). This form of feedback is similar to the feedback that we used in our work on 
mathematics, and that our research suggests can be an effective stimulus for guided inquiry and revision 
by students. 

SMART Lab and Kids Online are accessed next. Teachers use these sections to engage their classes in 
critical discussions. In the process, teachers and students discover how class members are thinking about 
issues. For example, SMART Lab summarizes the catalog choices and justifications submitted by each 
class and displays these data with the same data aggregated across all classes. Classes can discuss how 
their data are similar and different from the distributed class' data. Or they can discuss whether they 
agree with the most popular reasons given for choosing catalog items. In Kids Online, student-actors 
discuss their own catalog choices. We purposefully include reasoning errors in these presentations, and 
target common misconceptions that students have about river science. In the course of discussing Kids 
Online, students confront and hopefully debunk these misconceptions. 

After visiting the internet, student work to revise their catalog selections. They have opportunities to 
look up text-based resources that provide more in-depth information about science content relevant to 
various choices. For example, the text resources explain the need to break macroinvertebrates into 
categories, namely, pollution tolerant, somewhat pollution intolerant, or pollution intolerant. Students 
can use this information to understand why the TetraBen Laser Counter, which counts all 
macroinvertebrates but does not sort them, does not provide the kind of data they would need. 

After revising their thinking, students again visit the SMART WWWeb and make new choices of 
catalog items and new justifications. They can then see summarized data from their class and other 
classes and see how the data have changed. Following the correct choice of an appropriate object (in this 
case the .5mm Hochmeister Kick Net), students work with a CD-ROM simulation that allows them to 
'see' a sample of macroinvertebrates, calculate a water quality index, and compare their results with 
baseline data from previous years. Although each student gets a different sample of macroinvertebrates, 
each set of data shows that there is a serious absence of pollution sensitive macroinvertebratesohence 
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something is wrong. 

The SMART Challenge continues by next having students choose items for doing an oxygen test. Again, 
they make their choices via the Web and see data that summarizes the choices of other classes. Also, 
students gain access to text-based resources (which sometimes reside in other internet sites) that help 
them understand the science underlying various choices. And they eventually do some experiments on 
their own. For example, students are encouraged to test the amount of dissolved oxygen in a tank of 
water prior to putting fish in it and after the fish have lived in it for at least one day. With appropriate 
testing instruments, data show that there is less dissolved oxygen in the water after the fish have been 
there. For classrooms that cannot do actual tests, simulated, computer-based tests are available. All of 
these activities are preliminary to project-based activities in which students conduct water quality testing 
at a local river and present and publish their findings for the local Water Quality Management 
Department. 

As noted above, our work with SMART WWWeb challenges is just beginning: The Stones River 
Mystery challenge is the first we have attempted. By providing students and teachers with frequent 
opportunities for formative assessment and revision, we believe that we can better help them reach the 
goals of the National Standards in areas such as science and mathematics. Our plans are to create 
additional internet-based challenges that focus on 'big ideas' in areas such as mathematics, social studies, 
and literature. 
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